Skip to content

Resolve struct field types and byte offsets in the parsed IDL#45

Merged
estebanzimanyi merged 1 commit into
MobilityDB:masterfrom
estebanzimanyi:fix/parser-resolve-struct-layouts
Jul 10, 2026
Merged

Resolve struct field types and byte offsets in the parsed IDL#45
estebanzimanyi merged 1 commit into
MobilityDB:masterfrom
estebanzimanyi:fix/parser-resolve-struct-layouts

Conversation

@estebanzimanyi

@estebanzimanyi estebanzimanyi commented Jul 10, 2026

Copy link
Copy Markdown
Member

The catalog derives its struct layouts from the installed MEOS headers, giving FFI consumers (#[repr(C)] structs, cgo/cffi field access) accurate field types and byte offsets.

Where the layout comes from. The install step writes the generated, self-contained headers: meos_export.h splices postgres_ext_defs.in.h in place of the source tree's #include <postgres.h>. Parsing those headers, libclang resolves each field to its real C type and byte offset instead of degrading it to int at offset -1.

provision-meos. On the build-libmeos: true path the action builds libmeos and stages the MEOS headers into a dedicated prefix, then derives meos-idl.json from that prefix. A shared prefix such as /usr/local/include carries unrelated system headers (OpenSSL, PostgreSQL) that the header glob would pull into the catalog; the dedicated prefix holds only MEOS headers. The catalog-only path (build-libmeos: false) keeps the source-tree parse.

Parser. _clang_extra_args adds the compiler's own system include search path (from clang -E -v) so <stdbool.h> / <stddef.h> / size_t resolve — the pip libclang wheel ships no configured search path. size_t is force-included because meos.h uses it without an explicit #include. extract_struct runs field types through _c_spelling, normalising clang's _Bool keyword to the catalog's bool spelling.

Span: spantype:uint8@0, basetype:uint8@8, lower_inc:bool@16, upper_inc:bool@24,
      padding:char[4]@32, lower:Datum@64, upper:Datum@128

CI. The tests workflow builds all-families libmeos through the action, derives the catalog from the staged headers, and runs the suite. tests/test_struct_layout.py asserts the Span / STBox / TInstant layouts, that no field carries the _Bool keyword, and that every CORE-family struct field has a resolved offset.

@estebanzimanyi estebanzimanyi force-pushed the fix/parser-resolve-struct-layouts branch 2 times, most recently from 0bda23c to a47cc0b Compare July 10, 2026 23:04
Struct field types and byte offsets in meos-idl.json come out FFI-accurate when
run.py parses the installed MEOS headers instead of the source tree. `make
install` writes the generated, self-contained headers — meos_export.h splices
postgres_ext_defs.in.h in place of the source's `#include <postgres.h>` — so
libclang resolves the bool / uint8 / Datum fields and their real byte offsets
rather than degrading every field to int at offset -1 (unusable for the FFI
bindings' #[repr(C)] structs and cgo/cffi field access).

- The provision-meos action, on the build-libmeos path, stages the MEOS headers
  into a dedicated prefix and derives the catalog from there. A shared prefix
  like /usr/local/include carries unrelated system headers (OpenSSL,
  PostgreSQL) that run.py's header glob would otherwise pull into the catalog;
  the dedicated prefix holds only MEOS headers. The catalog-only path keeps the
  source-tree parse.
- parser.py adds the compiler's own system include search path (from
  `clang -E -v`) so <stdbool.h> / <stddef.h> / size_t resolve; size_t is
  force-included because meos.h uses it without an explicit include.
- extract_struct runs field types through _c_spelling, normalising clang's
  _Bool keyword to the catalog's bool spelling.
- pytest.yml builds libmeos, derives the installed catalog, and runs the suite.
- tests/test_struct_layout.py guards the resolved Span / STBox / TInstant
  layouts.

Span resolves to spantype:uint8@0, basetype:uint8@8, lower_inc:bool@16,
upper_inc:bool@24, padding:char[4]@32, lower:Datum@64, upper:Datum@128.
@estebanzimanyi estebanzimanyi force-pushed the fix/parser-resolve-struct-layouts branch from a47cc0b to 6872fa9 Compare July 10, 2026 23:07
@estebanzimanyi estebanzimanyi merged commit 0e3774f into MobilityDB:master Jul 10, 2026
2 checks passed
@estebanzimanyi estebanzimanyi deleted the fix/parser-resolve-struct-layouts branch July 10, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant